Bilevel hyperparameter optimization for support vector classification: theoretical analysis and a solution method

Abstract Support vector classification (SVC) is a classical and well-performed learning method for problems. A regularization parameter, which significantly affects the performance, has to be chosen this usually done by cross-validation procedure. In paper, we reformulate hyperparameter selection problem support as bilevel optimization in upper-level minimizes average number of misclassified data points over all folds, lower-level problems are $$l_1$$ l 1 -loss SVC problems, with each one fold T-fold cross-validation. The resulting model then converted mathematical program equilibrium constraints (MPEC). To solve MPEC, propose global relaxation algorithm (GR–CV) based on well-know Sholtes-type (GRM). It proven converge C-stationary point. Moreover, prove that MPEC-tailored version Mangasarian–Fromovitz constraint qualification (MFCQ), key property guarantee convergence GRM, automatically holds at feasible point MPEC. Extensive numerical results verify efficiency proposed approach. particular, compared other methods, our enjoys superior generalization performance almost sets used paper.